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ABSTRACT 

This paper discusses some of the analytical decisions 
that an investigator must make during the course of a life 
support system trade study. 

Equivalent System Mass (ESM) is often applied to 
evaluate trade study options in the Advanced Life 
Support (ALS) Program. ESM can be used to identify 
which of several options that meet all requirements are 
most likely to have lowest cost. It can also be used to 
identify which of the many interacting parts of a life 
support system have the greatest impact and sensitivity 
to assumptions. 

This paper summarizes recommendations made in the 
newly developed ALS ESM Guidelines Document and 
expands on some of the issues relating to trade studies 
that involve ESM. In particular, the following three points 
are expounded: 

1) The importance of objectives : Analysis objectives 
drive the approach to any trade study, including 
identification of assumptions, selection of characteristics 
to compare in the analysis, and the most appropriate 
techniques for reflecting those characteristics. 

2) The importance of results interpretation: The accuracy 
desired in the results depends upon the analysis 
objectives, whereas the realized accuracy is determined 
by the data quality and degree of detail in analysis 
methods. 

3) The importance of analysis documentation : 
Documentation of assumptions and data modifications is 
critical for effective peer evaluation of any trade study. 
ESM results are analysis-specific and should always be 
reported in context, rather than as solitary values. For 
this reason, results reporting should be done with 
adequate rigor to allow for verification by other 
researchers. 


INTRODUCTION 

The ALS ESM Guidelines document (Levri et al., 2003) 
was drafted to provide detailed instructional material for 
researchers who are performing ESM evaluations. The 
document was developed under the Systems Integration, 
Modeling and Analysis (SIMA) Element of the ALS 
Program. It provides a definition of ESM, describes how 
to calculate ESM, and discusses interpretation of ESM 
results. 

ESM is used as a measure of transportation cost in ALS 
trade studies. An ESM trade study may be performed to 
compare various technologies, hardware devices, 
hardware configurations and/or control approaches. 
Because the cost to transport a payload is proportional to 
the mass of that payload, a mass-based measure such 
as ESM is often used to quantify the cost of the life 
support system and associated infrastructure. An ESM 
value is the sum of the life support system mass and 
appropriate fractions of supporting system masses, 
including pressurized volume, power generation, cooling, 
and crewtime, for maintaining a specified crew over the 
duration of a specified mission. 

This paper does not discuss the ESM equation and 
parameter definition. (For the reader’s interest, the ESM 
equation and parameter definitions are provided in Levri 
et al., 2003.) Rather, this paper addresses three issues 
of interest that are considered in the ESM Guidelines 
Document: the importance of objectives, the importance 
of results interpretation, and the importance of analysis 
documentation. Clear objectives are necessary for 
identification of the questions of interest and related 
assumptions, identification of system characteristics to 
reflect in the analysis, and definition of the system to the 
appropriate extent and level of detail. Results 
interpretation and analysis documentation is critical 
because the author of the study is the person in the best 
position to evaluate the results, and the only one in full 
knowledge of all the details. Although these are 


important issues for all trade studies, this paper 
discusses that importance as it applies to trade studies 
that involve ESM in some manner. 

When it is feasible and appropriate, single metrics, such 
as ESM, can facilitate comparison. For ESM to be 
applied appropriately, the trade study options must first 
meet some common prerequisites. Only options that 
satisfy those prerequisites should be included in the 
study. After the trade options have been identified, some 
characteristics may require comparison, using ESM. 
However, it may not be possible to normalize all 
parameters in a study to a single ESM value. For 
example, when buying a car, most considerations can be 
reduced to some metric like cost per passenger mile; 
however, issues such as aesthetic appearance and 
safety may not be easily lumped into that same metric. 
Thus, some characteristics can either be examined as 
prerequisites, or compared in some other manner. 

THE IMPORTANCE OF OBJECTIVES 

Analysis objectives should drive all facets of the ESM 
computation. Objectives should be clearly defined in 
order for the investigator to determine the mission of 
interest and system characteristics to capture in the 
study, to define the appropriate system, and to 
appropriately apply data. 

Analyses should address important questions. Important 
questions are generally ones that make a difference to 
decisions that need to be made in the near future. 
Currently, there is a lot of interest in technology selection, 
thus that is an appropriate area to investigate. However, 
the technologies must be significantly different for the 
answer to be of much value. For example, if two 
technologies make a difference to the overall mission 
cost of 1%, it makes little difference which one is 
selected. 

The quantity of interest in an ESM analysis is 
comparison of the total system impact of trade options, 
which is often more complicated than simple accounting 
of hardware items. Determination of the total system 
impact involves defining the system of interest to the 
appropriate extent and level of detail to comprehensively 
capture cost impacts of trade options. In order to capture 
the cost impacts of the trade options on the entire life 
support system, the analyst must consider the important 
interfaces to the system of interest. This concept applies 
to all systems involved in a mission, even if some costs 
are traditionally considered to be out of the scope of the 
life support system. 

Although technically not illustrative of ESM, some 
computations other than the total system impact can be 
interesting. For example, it may be useful to know that 
the equipment mass of one trade study option is 30% 
heavier than an alternative, even if that computation 
does not reflect the total system impact. 


Ideally, analysis objectives are defined at the inception of 
the study, in an appropriate level of detail. However, the 
need for further clarification of the objectives often arises 
during the course of the research. Consequently, a trade 
study can be iterative. With experience, an analyst can 
gain foresight into the proper level of detail that is 
required in defining the study objectives. As the details of 
trade options emerge during the study, the objectives of 
the study may require re-clarification. 

Study objectives drive decisions on the mission of 
interest, selecting characteristics to reflect in the 
analysis, defining the appropriate system, and applying 
data appropriately, as discussed below. 

IDENTIFYING THE MISSION OF INTEREST AND 
RELATED ASSUMPTIONS 

The results of an ESM analysis depend on the 
assumptions made about the operating environment, the 
subsystem of interest, and the surrounding system. 
Consequently, an ESM analysis must be done with a 
particular mission and set of assumptions in mind. 

The ALS Reference Missions Document (Stafford et at., 
2001) is one source that analysts can use for selection of 
a particular reference mission for consideration in a trade 
study. Indeed, if the mission of interest is addressed in 
the ALS Reference Missions Document (RMD), it is 
recommend that the RMD assumptions be used in the 
study baseline. If not, these missions can be used as a 
starting point and changes to that mission can be 
documented. 

When making trade comparisons, the analyst must 
consider the suitability of each trade study option in a 
mission. For example, if comparing two technologies, the 
functions that the technology performs should be 
desirable for that mission scenario. As an explicit (and 
relatively obvious) example, it would be inappropriate to 
compare two soybean-processing devices in reference to 
a mission in which food is entirely prepackaged and 
ready to eat. (In such a scenario, a food-processing 
device would never be needed.) However, the suitability 
of trade options in particular missions is not always clear. 

In fact, the objectives of a trade study may actually be to 
identify the appropriateness of a trade option to a 
particular mission, based upon the total system impact of 
that option. For example, a valid study could consider 
the cost of growing soybeans to provide the crew with 
soybean-based fresh foods on a mission, compared to 
not growing soybeans and providing the crew with other 
food types. 

Top-level assumptions related to the missions in the ALS 
Reference Missions Document are documented in the 
ALS Baseline Values and Assumptions Document 
(BVAD) (Hanford 2002). The BVAD also identifies 
possible assumptions for missions that are not included 
in the RMD. Such assumptions include, for example, the 


number of crew members, number of visits to each site, 
mission duration, habitable volume, infrastructure costs, 
crewmember body mass, and typical metabolic loads. 
When deemed applicable by the analyst, the values 
provided in the BVAD may be applied to trade studies. 
(One motivation for developing the BVAD is to provide 
guidance to the ALS Community on reasonable 
assumptions for various missions. Use of a common set 
of assumptions facilitates identification of the reasons for 
differences in trade study results.) 

In particular, infrastructure costs, or “equivalency factors” 
drive the relative impacts of mass, volume, power, 
cooling and crewtime needs on the computed cost. As a 
result, the analyst should make efforts to use the most 
appropriate and reliable information for the equivalency 
factors. With continued research, collection of more 
accurate data will hopefully result in equivalency factors 
of greater accuracy in the BVAD. 

In addition to the top-level mission assumptions, notions 
about the details of system hardware, configuration and 
control are inherently made throughout a trade study. As 
the analyst gains knowledge of the details of operation of 
the various subsystems within the system of interest, 
initial assumptions may require revision. All assumptions 
(both top-level and more detailed) should be described 
(and well organized) throughout the trade study 
documentation. As this can be quite arduous, effort 
should be focused on the issues that most affect the 
results. 

SELECTING SYSTEM CHARACTERISTICS TO 
REFLECT IN THE ANALYSIS 

Based upon the analysis objectives, the analyst must 
determine which characteristics should be captured in 
the trade study. Characteristics of interest might be 
based upon function, availability, safety, gravity 
dependence, radiation susceptibility, noise levels, or a 
variety of other attributes. This determination will depend 
on the particular study, and often an initial trade study 
needs to be made in order to develop a methodology and 
identify what assumptions or data are needed. Thus, it 
can be necessary to iterate analysis steps to 
appropriately represent the most significant 
characteristics. 

The investigator must then determine the methods by 
which those characteristics will be reflected. 
Characteristics of interest may be considered 
prerequisites for inclusion in the study, and/or they may 
be compared (quantitatively and/or qualitatively) between 
trade study options. Note that qualitative comparisons 
should only be used when there is inadequate data 
available for a quantitative comparison. There is a 
danger in making biased decisions based on qualitative 
comparisons of critical issues. 


For an example of prerequisite characteristics, consider 
an analysis whose objective is to identify the lowest ESM 
technology that provides a certain level of moisture 
content reduction (water activity) in wasted food 
materials. Thus, only technologies that can provide that 
specific function, at the required level of moisture 
removal (or greater) should be considered in the 
analysis. 

Notice that the objective in the prior example strongly 
drives the analysis approach. For example, if the reason 
for removing the water from the wasted food is to 
stabilize the food materials before being stored as 
wastes, then the important prerequisite is that the water 
be removed down to a certain water activity from that 
material. However, consider the case where water 
removal from the wasted food was intended to not only 
stabilize the waste but also to recover water for later 
consumption by the crew. In that case, the objectives of 
the study may be to determine the lowest ESM option 
that removes water to a certain level and at a certain 
purity. Also notice that the objectives of the prior 
example were to “identify the lowest ESM technology...”. 
By declaring this in the objective, the study considers 
power needs, logistics, crewtime, etc., not just moisture 
content removal, and may address issues such as 
alternative sources of water, such as availability of water 
from local materials. 

However, comparable function might not be the only 
required characteristic. For example, if the analysis is 
being done to evaluate technologies for water recovery 
from wasted food for a low-Earth orbit mission, then any 
implementations 1 of technologies that are incompatible 
with microgravity should be eliminated from the study. 
Similar prerequisites may exist for performance, 
availability, safety, radiation susceptibility, noise levels 
and other characteristics, depending on the analysis 
objectives and the investigator’s judgement. 

The prerequisites for some characteristics may be 
exceeded in some trade options. Generally, there is no 
value in exceeding the performance requirements. 
However, depending on the objectives of the study, in 
some cases, the benefits (or detriments) of exceeding 
those prerequisites should be quantified or qualified. In 
the example in the previous paragraph, the water 
recovery efficiency (percentage of water recovered) may 
be greater in one trade option than in another. The 
difference (if in fact beneficial or detrimental) of 
exceeding this requirement might be reflected in an ESM 


1 Most technologies are not microgravity sensitive. 
However, particular implementations of technologies are 
often microgravity sensitive. For example, a hardware 
device that uses gravity flow might be fitted with 
appropriate pumps to make it microgravity compatible. 
Thus, it may be important for an analyst to consider 
alternative implementations of a technology in a trade 
study. 


analysis, by some other quantitative means, or in a 
qualitative manner in the study. 

Some characteristics that are not prerequisites in the 
trade study may also require comparison. For example, 
assume that an analyst decides that comparable 
functionality is the only prerequisite for including a 
technology in an evaluation for a Mars surface mission. If 
the specific function required is water recovery from 
inedible plant materials, then many different 
technologies, including heat-drying, freeze-drying, 
composting, and incineration may be included in the 
analysis. However, the above-mentioned options may be 
very different in terms of availability, safety, radiation 
susceptibility, noise levels, waste stabilization 
capabilities, and other characteristics. If such differences 
are expected by the investigator to be important in 
selection between trade options, then those differences 
should be compared in the study. 

For any characteristics that will be compared, the analyst 
must decide upon a means of comparison. For a general 
example, after considering the type and quality of 
analysis data that is available, the analyst may conclude 
that an ESM evaluation would reflect differences in 
launch costs between two trade options, given functional 
and performance requirements. However, the analyst 
might conclude that availability, safety and other 
differences should not or cannot be reflected in the ESM 
evaluation. Such a decision may be made for reasons of 
data inadequacy, uncertainty about flight requirements 
for those characteristics, or the availability of a more 
appropriate quantification method. 

After determining the characteristics of interest and the 
means by which to capture those characteristics in the 
study, the system may be defined to the appropriate 
extent and level of detail. 

DEFINING THE SYSTEM TO THE APPROPRIATE 
EXTENT AND LEVEL OF DETAIL 

The analyst should define the system to the extent and 
level of detail necessary to capture the total system 
impact of the trade options. However, quantification of 
the “total” system impact often requires more resources 
(time and money) than are available for an analysis. In 
such cases, the analyst must use judgement to 
determine the most important characteristics to reflect, 
by the most effective and appropriate means. Again, the 
decisions made by the analyst must be driven by the 
specific study objectives. 

In general, the ESM for the entire life support system 
should be calculated. It may, however, only be 
necessary to calculate ESM for a portion of the life 
support system, if the rest of the system remains 
identical, for all ESM-quantified characteristics, between 
trade study options. Thus, the analyst must determine 
the characteristics that will be reflected by an ESM 


computation, and define the system accordingly, so that 
the system can be sized appropriately. Care should be 
taken that a significant interface has not been neglected. 

This section presents brief examples to illustrate system 
definition for ESM purposes. For the purposes of 
Advanced Life Support, the system extent may range 
from the entire life support system and interfaces to any 
subset thereof. 

The system should be defined to the appropriate extent. 
The analyst should consider any portion of the life 
support system that has a significant effect upon, or is 
significantly affected by the trade options. For example, 
consider a comparison between the total system impacts 
of growing 20m 2 of salad crop versus growing 20m 2 of 
potato crop. In addition to the differences in crop 
production specifics, the evaluation should include 
differences outside of the biomass production system 
that have a significant impact on study results. A proper 
comparison of the two options, would involve 
development of an entire dietary concept, and 
specification of how all parts of the diet would be 
satisfied. The potato crop option may also be more 
effective in generating oxygen, removing carbon dioxide 
and producing food energy than the salad crop option, 
resulting in sizing differences in air revitalization 
equipment and prepackaged food stores. Additionally, 
different quantities and compositions of nutrient solution 
may be needed for the different crop options, which may 
impact requirements for shipping or recycling of 
nutrients. Depending on the analyst’s assumptions about 
the role and operation of the water recovery system 
(WRS), the WRS may require resizing between trade 
options. The analyst’s decision of which subsystems to 
include when defining the system for study depends on 
the analysis objectives and the investigator’s judgement. 

The system should be defined to the appropriate level of 
detail. In the comparison of salad and potato crops, 
issues such as palatability differences and differences in 
crew psychological benefits between the two options may 
be difficult to reflect in the ESM computation. In that 
case, if considered by the analyst to be important, such 
differences should be addressed, qualitatively and/or 
quantitatively, elsewhere in the study, Some differences, 
such as the sensitivity of the different crop types to 
system perturbations, may be, in theory, quantifiable in 
the ESM computation. However, the data necessary for 
such quantification may not be available. If inadequate 
data exists for quantifying critical differences between 
options, then those differences should be discussed 
qualitatively in some other manner in the study. In order 
to minimize the number of characteristics that require 
investigation, a preliminary sensitivity analysis to identify 
the change in outcome per change in parameter values 
can aid in identifying the most critical study parameters. 

Regardless of the system definition approach, a 
justification of the system extent and level of detail 



should be provided in the analysis report, so that the 
analysis choices can be scrutinized. Such choices should 
be in harmony with the objectives of the analysis. 

APPROPRIATE APPLICATION OF DATA 

Raw data provided by researchers and technology 
developers may require some modification in order to fit 
into the context of a particular ESM evaluation. Data 
modifications may include, but are not limited to, 
adjustment for a different development state (notably 
flight ready), adjustment for environment, and/or system 
scaling. Although development state adjustment and 
system scaling are the most common types of data 
modification in an ESM analysis, other types of 
modification may be necessary. All data modifications 
should be explained and quantified in the analysis report. 

Adjustment for Development State 

Depending upon the objectives of the analysis, some 
data may require adjustment for the appropriate 
development state. In ALS trade studies, the flight-ready 
development state is typically of greatest interest for 
ESM evaluations. As an alternative to modifying data for 
a flight-ready development state, an analyst may state 
assumptions that flight equipment is similar to some 
earlier design stage. Such assumptions are often 
necessary, since the data needed for development state 
adjustments is often unavailable. 

For trade study comparisons, all equipment data should 
be representative of comparable development states. 
For example, equipment in the infancy of development 
should not be compared with equipment for a different 2 
technology at a flight-ready state. Adjusting data to a 
future development state often requires that the 
investigator make assumptions in order to modify values 
or to estimate missing values. All such assumptions and 
their reasoning should be thoroughly documented so that 
the analysis results can be understood in the appropriate 
context. 

For example, in determining a flight-ready development 
state ESM, the analyst may predict that advances 
through R&TD will result in improved design 
sophistication such as material types, automation, and 
processor efficiency, thereby reducing the flight-ready 
ESM. Similarly, other critical characteristics may 
necessitate modification of the raw data to appropriately 
capture those qualities in the analysis. 

To adjust raw data for a flight-ready development state, 
in addition to improvements in the technology during 
development, the investigator should account for the 


2 However, for the purposes of forecasting changes in 
characteristics for a particular piece of equipment, it is 
valid to compare data for an early development state with 
data for a later development state. 


environment of the mission and any other anticipated 
mission requirements. 

For example, launch forces and the level of gravity 
during different phases of a mission will affect equipment 
structural strength requirements, and may drive 
modification of processes to accommodate differences in 
phase separation, convection and other characteristics. 
As another example, for a Mars surface mission, 
equipment exposed to the Mars surface may require 
modification for radiation conditions and operation during 
dust storms. Trade option assumptions may also need to 
accommodate constraints on the release of materials to 
the Martian environment due to planetary protection 
concerns. Additionally, in order to satisfy a requirement 
on the degree of risk to the crew equivalent to that of 
Shuttle or International Space Station missions, 
equipment for a Mars surface (or any longer-term, more 
distant mission) may require greater system reliability. 

Probably the most difficult issue in comparing flight ready 
options with developmental options is the amount of 
detail available. Flight equipment is by definition 
complete, thus necessary data is available. 
Developmental equipment, however, might have no data 
on maintenance costs or crew time, and may not have all 
the pieces necessary for safely operating the equipment 
in a space environment. For example, a bench-top 
electrolysis unit might not have all of the sensors and 
safety equipment required to prevent - a hydrogen 
explosion. It might not have been run long enough to 
know the lifetime of the consumables and spares. 

The analyst must judge the importance of development 
state adjustments in the reflection of trade study options 
against the availability of data and the analysis resources 
needed to make such modifications. Requirements of 
future missions are (by definition) uncertain, and the 
information needed to make data adjustments is often 
unavailable. For example, if a failure modes and effects 
analysis has not been conducted on a system, then the 
analyst will have little knowledge of the types of 
modifications needed to bring a system to a certain level 
of reliability. The analyst may also not have knowledge of 
the degree of reliability/availability that will be required for 
future missions. As time progresses and future missions 
come closer in time, some mission requirements will be 
developed. However, in the current state of mission 
uncertainty, an analyst must anticipate the requirements 
of future missions. Thus, an investigator may choose to 
qualitatively discuss necessary data alterations, rather 
than attempt to quantify the modifications. Regardless of 
the approach, all data modifications deemed appropriate 
for the study should be clearly explained in the analysis 
documentation. 

System Sizing and Data Scaling 

The analyst must determine the proper hardware sizes 
for the mission scenario of interest for the system that 


has been deemed appropriate for the study. Hardware is 
commonly sized according to a characteristic parameter. 
Lacking other information, throughput, or mass flowrate 
through the equipment, is often a good correlating 
parameter. For example, in sizing an oxygen generation 
device, a typical approach is to multiply the number of 
crewmembers by the historical, individual, average 
oxygen demand. 

The level of detail necessary for defining the system of 
interest determines the level of detail necessary in the 
system-sizing effort. In this respect, the investigator’s 
judgement should be used to determine which material 
compounds should be included in the sizing effort. For 
example, if the analyst deems it necessary to compare 
mineral requirement differences in biological systems, 
then the sizing effort must consider the minerals of 
interest. However, if a material compound has a 
relatively minor effect on system sizing, then it may be 
appropriate to exclude that particular compound from the 
sizing effort. 

Depending on the objectives of the analysis, the 
approach to system sizing may range from steady-state 
to transient conditions. In other words, system sizing 
efforts may lie anywhere in between (or include) these 
two approaches. A steady-state mass balance requires 
less analysis effort but is less realistic than the more 
effort-intensive transient mass balance. Steady-state 
mass balances can be easily implemented using 
spreadsheet software, whereas transient mass balances 
are most easily performed through the use of transient 
simulation tools. If batch or semi-batch processors exist 
in the system, an alternative between resource-intensive 
transient simulation and (possibly) inaccurate steady- 
state estimates may be necessary. In such a case, the 
analyst might choose to modify steady-state flowrates to 
agree with the batch or semi-batch nature of the 
processors. Thus, if a steady-state mass balance is used 
but the analyst knows that an incinerator (for example) 
will actually be operating in semi-batch mode, then the 
analyst may modify the mass balance results for the 
incinerator to reflect that knowledge. This may mean 
estimating modifications for the mass flowrate of 
materials into the semi-batch mode incinerator and for 
the material containment that is needed prior to 
incineration. 

In this same vein, the investigator should decide whether 
to size the system for solely nominal (no fault) operation, 
or if the system should be sized for specific off-nominal 
events (faults). Thus, system sizing possibilities span a 
space bounded by 1) steady-state sizing with nominal 
operation, 2) steady-state sizing with off-nominal events, 
3) transient sizing with nominal operation, and 4) 
transient sizing with off-nominal events. For each 
individual study, the analyst’s judgement should be used 
to determine the most appropriate sizing tactics. The 
most common approach to date for system sizing efforts 


has been steady-state calculations of daily loads, under 
nominal operation. 

Once the appropriate sizing parameter values have been 
determined, the investigator should scale the hardware 
raw data to those values. For example, a technology 
developer may provide hardware raw data for a 
laboratory prototype that processes only a fraction of the 
flow that would require processing in an actual mission 
or test bed. Thus the hardware data must be scaled up 
to accommodate the desired mass flowrate (the sizing 
parameter). An industrial process, on the other hand, is 
likely to be much larger than needed for space missions 
and must be scaled down. If an objective of the analysis 
is to compare functionally similar processors for a 
specific mass throughput (or other sizing parameter), 
raw data scaling is necessary. 

A simple approach is to scale raw data linearly with 
respect to the sizing parameter. However, more accurate 
scaling might be obtained by using scaling factors, which 
are not necessarily linear, provided by the technology 
developer or by using component-specific scaling factors 
that are standard to the chemical industry. (The reader is 
referred to Yeh, et. al., 2001, for examples of scaling 
factors used in the Chemical Engineering Industry.) 

As a simplification, the entire piece of hardware can be 
scaled to the same sizing parameter. However, because 
there can be components of a piece of hardware (e.g. 
controls logic) that are not dependent on a sizing 
parameter (such as mass flowrate), more accurate sizing 
parameters should be used for those components, if 
readily available. Whatever the approach, sizing 
parameter values, scaling factors and associated 
assumptions should be adequately explained in the 
analysis documentation. 

THE IMPORTANCE OF RESULTS 
INTERPRETATION 

To legitimately implement ESM results in a selection 
process, the results must be more accurate than the 
degree of separation between option results. Thus, a 
rough calculation may be adequate to rank two options if 
the result values are grossly different from each other. In 
other cases, a high level of accuracy is required to make 
comparisons of options that have very similar ESM 
values. If the results of an ESM comparison are too close 
to make a judgement, then both options may be equally 
good from the ESM perspective, and other issues (such 
as TRL, robustness and/or simplicity/complexity) are 
likely to drive technology selection. 

The desirable degree of results accuracy can depend 
upon the analysis objectives. Because of the 
compounding nature of error in analyses, if study 
objectives necessitate a very detailed analysis involving 
large amounts of data, then a high degree of accuracy in 
data and methods is necessary. However, if the 



objectives necessitate an analysis of a low level of detail, 
then the required degree of data and methods accuracy 
might be reduced. 

Researchers that provide data for an ESM evaluation 
may execute various levels of rigor in the data collection 
process. In addition, if data is obtained from 
documentation, rather than directly from a researcher, 
error propagation in data values can be a concern. On 
the other hand, data from sources with a very rigorous 
review process can be quite reliable. Consequently, an 
analyst may be faced with data from a range of sources 
and various degrees of accuracy. Because the accuracy 
of data feeds into the confidence of ESM results, the 
investigator should have at least a basic impression of 
data quality. Confidence in results is also affected by 
decisions affecting the rigor of development state 
adjustments and scaling efforts. The analyst should 
balance the degree of effort needed for adequate results 
confidence with resource availability during each step of 
the ESM evaluation process. 

The degree of results separation that is necessary to 
conclude a significant difference (i.e. judge one option 
preferable over another) depends upon the degree of 
confidence in the data used and the degree of accuracy 
in the analysis methods. Data confidence and analysis 
accuracy is analysis-specific. Therefore, the degree of 
results separation that is required to declare one option 
preferable to -••-another is also analysis-specific. The 
investigator’s judgement should be used to determine 
the analysis-specific necessary degree of results 
separation. 

If data accuracy is uncertain, a reasonable assumption is 
that an order of magnitude of separation (i.e. factor of 
10) between ESM results is adequate to conclude a 
significant difference between options. A difference of a 
few percent is likely to be sensitive to assumptions. 
However, the degree of confidence in technology data 
generally improves as the technology is further 
developed. Thus, ESM comparisons of trade options at 
advanced development stages may require less of a 
degree of separation between results in order to declare 
a significant difference between options. 

THE IMPORTANCE OF ANALYSIS 
DOCUMENTATION 

All assumptions (both top-level and more detailed), input 
data, sources, and analysis methods should be 
described (and well organized) throughout the trade 
study documentation. As this can be quite arduous, 
effort should be focused on the issues that most affect 
the results and add value to the final report. 

Sometimes the greatest value of an analysis can be the 
identification of superior or inferior input data and their 
sources. In this same vein, unreferenced data values can 
make a calculation suspect. 


ESM results are analysis-specific and should always be 
reported in context, rather than as solitary values. For 
this reason, results reporting should be done with 
adequate rigor to allow for verification by other 
researchers. All input data, sources, analysis 
approaches and critical assumptions made on the 
hardware, configuration, and control approaches in the 
system should be clearly stated in reporting the 
evaluation results, so that the results can be considered 
in context. Even if the ALS BVAD is used to determine 
values for an analysis, the analysis documentation 
should identify which values were applied and explain the 
implied assumptions. 

Some of the decisions that must be explained in the 
results documentation include the following material, and 
all associated assumptions, as a minimum: 

1 . Description of analysis objectives. 

2. Explanation of characteristics deemed worthy of 
examination (e.g. critical characteristics). 

3. Explanations of which (if any) critical characteristics 
are captured by ESM and which are reflected by 
some other quantitative or qualitative means. 

4. Justification of the system extent and level of detail 
chosen for the ESM evaluation and other 
evaluations. 

5. Input data values, their sources, and a brief 
explanation of those values implemented. 

6. Justification of development state adjustments, 
sizing methods, and scaling factors, including 
quantification and references. 

7. A discussion of the expected accuracy of ESM (and 
other) results and associated interpretation. 

CONCLUSION 

This paper discussed the importance of some of the 
analytical decisions that an investigator must make 
during the course of a trade study. When applied 
appropriately, an ESM analysis can be a useful tool for 
making choices between trade study options. However, 
in cases where no conclusive decision can be drawn 
from an ESM analysis, the results may be useful in 
determining the most important cost drivers (mass, 
volume, power, cooling or crewtime) for a trade study 
option. The analysis effort can also draw attention to the 
need for possibly critical data that needs to be collected 
and/or verified. 
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DEFINITIONS, ACRONYMS, ABBREVIATIONS 

ALS: Advanced Life Support 

BVAD: Baseline Values and Assumptions Document 

ESM: Equivalent System Mass 

RMD: Reference Missions Document 

R&TD: Research and Technology Development 

SIMA: Systems, Integration, Modeling and Analysis 

WRS: Water Recovery System 



